From: Øyvind Kolås Date: Tue, 27 Nov 2018 02:37:10 +0000 (+0100) Subject: babl: decrease tolerance X-Git-Tag: archive/raspbian/1%0.1.106-3+rpi1^2~15^2~12^2~39 X-Git-Url: https://dgit.raspbian.org/%22http://www.example.com/cgi/%22/%22http:/www.example.com/cgi/%22?a=commitdiff_plain;h=5e40364b3953f90051e290dd185b91e64aea1a66;p=babl.git babl: decrease tolerance The following conversion is undesirable, going via linear u16 on the way to a different TRC u16 amount of quantization should be considered bad enough to be avoided. chosen RGBA u32 to R'G'B'A u16: steps: 4 error: 0.000005 cost: 876.000000 /usr/local/lib/babl-0.1/u32.so 0: RGBA u32 to RGBA u16 /usr/local/lib/babl-0.1/gggl-table-lies.so 0: RGBA u16 to RGBA float /usr/local/lib/babl-0.1/sse2-float.so 0: RGBA float to R'G'B'A float /usr/local/lib/babl-0.1/gggl.so 0: R'G'B'A float to R'G'B'A u16 This is slower but more correct: chosen RGBA u32 to R'G'B'A u16: steps: 3 error: 0.000000 cost: 1051.000000 /usr/local/lib/babl-0.1/simple.so 0: RGBA u32 to RGBA float /usr/local/lib/babl-0.1/sse2-float.so 0: RGBA float to R'G'B'A float /usr/local/lib/babl-0.1/gggl.so 0: R'G'B'A float to R'G'B'A u16 --- diff --git a/babl/babl-fish-path.c b/babl/babl-fish-path.c index c3a2289..2524bb4 100644 --- a/babl/babl-fish-path.c +++ b/babl/babl-fish-path.c @@ -21,7 +21,7 @@ #include "babl-internal.h" #include "babl-ref-pixels.h" -#define BABL_TOLERANCE 0.000005 +#define BABL_TOLERANCE 0.0000047 #define BABL_MAX_COST_VALUE 2000000 #define BABL_HARD_MAX_PATH_LENGTH 8 #define BABL_MAX_NAME_LEN 1024